In The Claims: 



1. (Original) A method for querying stored multimedia data in a computer system, 
comprising: 

receiving into an intermediate level a high-level concept from a user describing data 

to be retrieved; 

translating, in said intermediate level, said high-level concept into low-level queries 
by using system pre-defined high-level concepts; and 

transferring said low- level queries to a low level comprising one or more search 
engines; said one or more search engines performing a query of the stored multimedia information 
using said low-level queries. 

2. (Original) The method of claim 1 wherein said intermediate level comprises: 

a set of library modules, said set of library modules comprising: 
a concept library module for storing concepts; 

one or more library modules adapted to store said data from said one or more data 

sources; 

a cataloger module adapted to construct a new concept from said high-level concept 
using data from said concept library and library modules, thereby creating a concept construct, and 
to pass said concept construct to said concept library module for storage as a concept; and 

an interpreter module adapted to translate said high-level concept into low- level 
queries using said concepts stored in said construct library and to pass said low- level queries to said 
one or more search engines. 
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3. (Original) The method of claim 2 wherein said set of library modules further comprises 
at least one library module selected from the group comprising: 

a feature library module adapted to store multimedia features; 

a matching algorithm library module adapted to store matching algorithms; and 

a constraint library module adapted to store feature constraints. 

4. (Original) The method of claim 3 wherein each said library module further comprises an 
application program interface to receive said data from a said data source. 

5. (Original) The method of claim 3 wherein said cataloger module further performs the 

steps of: 

selecting a set of concept features from said feature library module; 
selecting a set of concepts from said concept library module for use as child- 
concepts; and 

selecting a set of constraints on said child concepts from said constraint library 

module. 

6. (Original) The method of claim 1 wherein said each said concept comprises a triplet of 
a set of child-concepts, a set of features, and a set of relationships. 

7. (Original) The method of claim 6 wherein said concepts comprise a hierarchical fuzzy 

graph data tree- structure comprising nodes, aggregation edges, and association edges and wherein: 
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said nodes correspond to said concepts and said features; 

said aggregation edges correspond to parent-child relationships; and 

said association edges correspond to said constraints. 

8. (Original) The method of claim 7 wherein said edges are weighted. 

9. (Original) The method of claim 3 further comprising a matching algorithm comprising 
GetNextMatch(), AssignNextMatch(), and ShiftNextMatchO procedures, wherein: 

said GetNextMatchO procedure comprises the steps: 

testqueue: if ^ewe.EmptyO; 

return NULL; 

head - >queue.Pop(); 

if tettiCompleteQ; 

return bead; 

head2 - >head.CopyQ; 

head2. AssignNextMatchO ; 

if/x^.Valid(); 

queue.Push(head2) ; 

^^ShiftNextMatchO ; 

queue .Push(head) ; 

Goto testqueue; 

said AssignNextMatchO procedure comprises the steps: 

child- >GetNextUnassignedO; 
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J child.matchj?tr + +; 

^ if (child.imtch_ptr} == NULL), then; 

child.match_ptr - >child.GetNextMatch(); 
Make child an assigned node; 
said ShiftNextMatchO procedure comprises the steps: 
Child - > GetNextUnassignedO ; 
child.match_ptr + +; 
if (child.match_ptr == NULL), then; 

child.matchjptr - >child.GetNextMatchO; 
wherein variables beady head2, and drild, all correspond to concept nodes; variable qmte 
denotes a priority queue of the corresponding concept node; and rmtch_ptr\s a pointer to the next 
possible match for a given concept node; PopO is a method to get the next node off the priority 
queue; Push0 is a method to put a node on the priority queue; EmptyO ^ a method to check if the 
priority queue is empty, GopyO is a method to copy a node; CompleteO is a method to check if the 
children assignment is complete; ValidO is a method to check if the children assignment meets the 
constraints; and GetNextUnassignedO is a method to select a variable that is unassigned. 

10. (Currendy Amended) A program storage device readable by machine, tang ibly 
embodying a program of instructions executable by the machine to perform method steps for a 

matching algorithm, said method steps comprising A matching algorithm comprising 

GetNextMatchO, AssignNextMatchO, and ShiftNextMatchO procedures, wherein: v 
said GetNextMatchO procedure comprises the steps: 
testqueue : if queueE mptyO ; 




return NULL; 
head - >queue.Pop(); 
if ^e^Complete(); 

return head; 
head2 - >head.GopyQ; 
head2. AssignNextMatchO ; 
if/eafi.ValidO; 

queue Push(head2) ; 
^az^ShiftNextMatchO ; 
queue .Push(head); 
Goto testqueue; 
said AssignNextMatchO procedure comprises the steps: 
child - > GetNextUnassignedO ; 
child.match_ptr + +; 
if (child.match_ptr} == NULL), then; 

child.match_ptr -- >child.GetNextMatchO; 
Make child an assigned node; 
said ShiftNextMatchO procedure comprises the steps: 
Child - > GetNextUnassignedO; 
child.matchjptr + +; 
if (child.match_ptr == NULL), then; 

child.match_ptr >child.GetNextMatchO; 



6 




wherein variables heady head2 y and child, all correspond to concept nodes; variable qmte 
denotes a priority queue of the corresponding concept node; and mttdo_ptr\s a pointer to the next 
possible match for a given concept node; PopO is a method to get the next node off the priority 
queue; Push() is a method to put a node on the priority queue; EmptyO is a method to check if the 
priority queue is empty Gopy0 is a method to copy a node; Complete 0 is a method to check if the 
children assignment is complete; ValidO is a method to check if the children assignment meets the 
constraints; and GetNextUnassigned() is a method to select a variable that is unassigned. 

11. (Original) A program storage device readable by machine,, tangibly embodying a 
program of instructions executable by the machine to perform method steps for querying stored 
multimedia data, said method steps comprising: 

receiving into an intermediate level a high-level concept from a user describing data 

to be retrieved; 

translating, in said intermediate level, said high-level concept into low- level queries 
by using system pre-defined high-level concepts; 

transferring said low- level queries to a low level comprising one or more search 
engines; said one or more search engines performing a query of the stored multimedia information 
using said low- level queries. 

12. (Original) The apparatus of claim 11 wherein said intermediate level comprises: 

a set of library modules, said set of library modules comprising: 
a concept library module for storing concepts; and 
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one or more library modules adapted to store said data from said one or more data 

sources; 

a cataloger module adapted to construct a new concept from said high-level concept 
using data from said concept library and library modules, thereby creating a concept construct, and 
to pass said concept construct to said concept library module for storage as a concept; and 

an interpreter module adapted to translate said high-level concept into low- level 
queries using said concepts stored in said construct library and to pass said low- level queries to said 
one or more search engines. 

13. (Original) The apparatus of claim 12 wherein said set of library modules further 
comprises at least one library module selected from the group comprising: 

a feature library module adapted to store multimedia features; 

a matching algorithm library module adapted to store matching algorithms; and 

a constraint library module adapted to store feature constraints. 

14. (Original) The apparatus of claim 13 wherein each said library module further 
comprises an application program interface to receive said data from a said data source. 

15 (Original) The apparatus of claim 13 wherein said cataloger module further performs 
the steps of: 

selecting a set of concept features from said feature library module; 
selecting a set of concepts from said concept library module for use as child- 
concepts; and 
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selecting a set of constraints on said child concepts from said constraint library 

module. 

16. (Original) The apparatus of claim 11 wherein said each said concept comprises a triplet 
of a set of child-concepts, a set of features, and a set of relationships. 

17. (Original) The apparatus of claim 16 wherein said concepts comprise a hierarchical 
fuzzy graph data tree- structure comprising nodes, aggregation edges, and association edges and 
wherein: 

said nodes correspond to said concepts and said features; 

said aggregation edges correspond to parent-child relationships; and 

said association edges correspond to said constraints. 

18. (Original) The apparatus of claim 17 wherein said edges are weighted. 

19. (Original) The apparatus of claim 13 further comprising a matching algorithm 
comprising GetNextMatch(), AssignNextMatch(), and ShiftNextMatch() procedures, wherein: 

said GetNextMatchO procedure comprises the steps: 

testqueue: if queue.Empxy() ; 

return NULL; 

head - >queue.Pop(); 

if beadComp\ete(); 

return head; 
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head2 >head.Copy(); 

/aa^.AssignNextMatch(); 

ifte^.VaUdO; 

queue.Push(head2) ; 

/^ShiftNextMatchO ; 

queue .Push(head) ; 

Goto testqueue; 

said AssignNextMatchO procedure comprises the steps: 

child- >GetNextUnassignedO; 

chdd.match_ptr + +; 

if (child.match_ptr} == NULL), then; 

child.match_ptr -- >child.GetNextMatchO; 

Make child an assigned node; 

said ShiftNextMatchO procedure comprises the steps: 

Child- >GetNextUnassignedO; 

child.match_ptr + +; 

if (child.match_ptr == NULL), then; 

child.match_ptr - >child.GetNextMatchO; 

wherein variables heady head2, and chiH all correspond to concept nodes; variable queue 

denotes a priority queue of the corresponding concept node; and mttchj?tr\s a pointer to the next 

possible match for a given concept node; PopO is a method to get the next node off the priority 

queue; Push() is a method to put a node on the priority queue; EmptyO is a method to check if the 

priority queue is empty, CopyO is a method to copy a node; CompleteO is a method to check if the 
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children assignment is complete; ValidO is a method to check if the children assignment meets the 
constraints; and GetNextUnassignedQ is a method to select a variable that is unassigned. 
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